Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refresh metadata cache after failed package installation #3348

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

bajertom
Copy link
Collaborator

@bajertom bajertom commented Nov 8, 2024

Fixes #3277

When testing I found out that there is different behavior in dnf5 compared to dnf4, where a mere dnf makecache would be enough, but dnf5 needs the --refresh option as well (or dnf clean metadata or dnf clean expire-cache), see https://bugzilla.redhat.com/show_bug.cgi?id=2324177

Not sure about the Exception being too generic. Should I create a new one, something like InstallationError? And notify the user through logger as to why metadata --refresh is happening?

Pull Request Checklist

  • implement the feature

tmt/steps/prepare/install.py Outdated Show resolved Hide resolved
tmt/steps/prepare/install.py Outdated Show resolved Hide resolved
tmt/steps/prepare/install.py Outdated Show resolved Hide resolved
@bajertom bajertom force-pushed the tbajer-dnf-cache-error branch 2 times, most recently from c736cd9 to 793ac8a Compare November 14, 2024 16:46
@bajertom bajertom requested a review from happz November 19, 2024 10:12
@happz happz added area | package managers Changes related to implementations of package managers plugin | install The prepare plugin for installing packages labels Nov 19, 2024
@happz happz modified the milestones: 1.40, 1.39 Nov 19, 2024
except Exception:
# Refresh cache in case of recent but not updated change do repodata
self.guest.package_manager.refresh_metadata()
self._install()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether should capture the first exception as well - this way, the original one is lost, with all its data. How about something like this?

try:
    self._install()

except Exception as exc1:
    # We do not have any special handling for exceptions raised by the following code.
    # Wrapping them with try/except gives us a chance to attach the original exception
    # to whatever the code may raise, and therefore preserve the information attached
    # to the original exception.
    try:
        # Refresh cache in case of recent but not updated change do repodata
        self.guest.package_manager.refresh_metadata()
        self._install()

    except Exception as exc2:
        raise exc2 from ex1

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, that seems like an unnecessary loss of data - better to catch them both. I used your code, if that's alright:)

@bajertom
Copy link
Collaborator Author

/packit test

@bajertom bajertom requested a review from happz November 19, 2024 15:14
@psss psss modified the milestones: 1.39, 1.40 Nov 21, 2024
@happz happz added the ci | full test Pull request is ready for the full test execution label Nov 22, 2024
@happz
Copy link
Collaborator

happz commented Nov 28, 2024

@bajertom please, add a unit test for this new operation. It can be fairly trivial, and it would verify those commands work without crashing. See e.g. https://github.com/teemtee/tmt/blob/main/tests/unit/test_package_managers.py#L420 or other unit tests.

To run new unit tests, the following command should work (with dev environment activated, and let's say you'd call the test test_refresh_metadata):

$ hatch -e dev shell
$ pytest -vv --showlocals tests/unit/test_package_managers.py::test_refresh_metadata

@bajertom
Copy link
Collaborator Author

bajertom commented Dec 3, 2024

I've added unit tests covering the command. Since there was difference between cases in "Metadata C/cache C/created" in different distros but for the same package manager (yum) I've decided to just stick with "Metadata". Is this enough or should I catch a more specific part of the stdout in this case?

Same goes for apt - it depends on whether the system is up to date or not. It's either (debian test)

Hit:1 http://repolink reponame
Hit:2 http://repolink reponame
50 packages can be upgraded. Run 'apt list --upgradable' to see them.

before apt update or (ubuntu test)

Hit:1 http://repolink reponame
Hit:2 http://repolink reponame
All packages are up to date.

if the system is up to date, so I decided to go for packages as a keyword here.

However I am lost with rpm-ostree packager. On my VM I can run rpm-ostree upgrade --check, but the unit test throws this error

DEBUG    tmt:log.py:742 Run command: podman exec ec41e25298a91ed3dd829df46b889f4605f5b33862696297c3146063dd030f5a /bin/bash -c 'rpm-ostree upgrade --check'
INFO     tmt:log.py:742     cmd: rpm-ostree upgrade --check
DEBUG    tmt:log.py:742 environment
DEBUG    tmt:log.py:742 Command event: 2.91e-06 waiting for process to finish
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: error: Loading sysroot: exit status: 1
DEBUG    tmt:log.py:742 Command event: 0.1897 waiting for process completed
DEBUG    tmt:log.py:742 Command event: 0.19 waiting for stream readers
DEBUG    tmt:log.py:742 Command event: 0.1902 stdout reader done
DEBUG    tmt:log.py:742 Command event: 0.1903 stderr reader done
DEBUG    tmt:log.py:742 Command returned '1' (failure).

@happz
Copy link
Collaborator

happz commented Dec 3, 2024

I've added unit tests covering the command. Since there was difference between cases in "Metadata C/cache C/created" in different distros but for the same package manager (yum) I've decided to just stick with "Metadata". Is this enough or should I catch a more specific part of the stdout in this case?

LGTM. It's not perfect, which is true for other test cases in that file, we're fine. Success of the command itself is a good indication things went well.

However I am lost with rpm-ostree packager. On my VM I can run rpm-ostree upgrade --check, but the unit test throws this error

DEBUG    tmt:log.py:742 Run command: podman exec ec41e25298a91ed3dd829df46b889f4605f5b33862696297c3146063dd030f5a /bin/bash -c 'rpm-ostree upgrade --check'
INFO     tmt:log.py:742     cmd: rpm-ostree upgrade --check
DEBUG    tmt:log.py:742 environment
DEBUG    tmt:log.py:742 Command event: 2.91e-06 waiting for process to finish
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: System has not been booted with systemd as init system (PID 1). Can't operate.
INFO     tmt:log.py:742     err: Failed to connect to bus: Host is down
INFO     tmt:log.py:742     err: error: Loading sysroot: exit status: 1
DEBUG    tmt:log.py:742 Command event: 0.1897 waiting for process completed
DEBUG    tmt:log.py:742 Command event: 0.19 waiting for stream readers
DEBUG    tmt:log.py:742 Command event: 0.1902 stdout reader done
DEBUG    tmt:log.py:742 Command event: 0.1903 stderr reader done
DEBUG    tmt:log.py:742 Command returned '1' (failure).

Never seen this one before, and there are other rpm-ostree tests where I'd expect this to appear. No idea so far.

Can you rebase the PR? That should make the patch easier to review and play with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area | package managers Changes related to implementations of package managers ci | full test Pull request is ready for the full test execution plugin | install The prepare plugin for installing packages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tmt should handle error due to outdated dnf cache
4 participants